As an example of an image pickup element having a functional pixel having a specific function as a part of pixels constructing the image pickup element, such a technique that a phase difference detecting function of an object image is added to the image pickup element, so that a dedicated AF (autofocus) sensor is made unnecessary and a high-speed phase difference AF is realized, has been disclosed.
For example, in the Official Gazette of PTL1, in a light receiving element (pixels) as a part of an image pickup element, a pupil dividing function is attained by deviating a sensitive area of a light receiving portion for an optical axis of an on-chip micro lens. Those pixels are used as focus detection pixels and arranged between image pickup pixels at a predetermined interval, thereby performing a phase difference type focus detection. Since the portion where the focus detection pixels have been arranged corresponds to a defective area of the image pickup pixels, image information of the focus detection pixels is formed by interpolation using image pickup pixel information around the focus detection pixel.
In the Official Gazette of PTL2, a unit for forming image information at a position of focus detection pixels has: a generation unit for generating image data from the image information of the focus detection pixels by gain correction; a generation unit for generating image information from information of image pickup pixels around the focus detection pixels by interpolation; and a frequency component detection unit for detecting a spatial frequency component of an object image, wherein those generation units are switched in accordance with the detected spatial frequency component.
However, the foregoing conventional techniques have the following problems.
As disclosed in PTL1, if an image signal of the focus detection pixel is interpolated on the basis of image signals of pixels around the focus detection pixel, as for image data which is obtained from those peripheral pixels, there is a possibility that a value which is fairly away from an output of the image pickup pixel which ought to have inherently existed at its position is calculated.
In the case of receiving light of an object of a low spatial frequency, an image at the position of the focus detection pixel has a high continuity for an image at the position of pixels only for image pickup, arranged around the focus detection pixel. Therefore, it is desirable to interpolate the image data at the position of the focus detection pixel on the basis of the image signals of the peripheral pixels. In this case, since the spatial frequency of the object image is low, a deterioration in sharpness caused by the interpolation processing is inconspicuous.
In the case of receiving light of an object of a high spatial frequency, since an image at the position of the focus detection pixel has a less continuity for the image at the position of the pixels only for image pickup, arranged around the focus detection pixel, a result of the interpolation processing differs largely depending on a selection of interpolation reference pixels.
In the case of receiving light of an object in which the continuity of the image is low in a specific direction, it is desirable that the pixels only for image pickup arranged in the direction in which the continuity of the image is high for the position of the focus detection pixel are selected as interpolation reference pixels. For an image in which the continuity of the image is low in all directions, it is difficult to estimate the image signal at the position of the focus detection pixel from the image signals of the peripheral pixels.
According to the conventional technique disclosed in PTL2, in the case of receiving light of an object of a high spatial frequency, image information at the position of the focus detection pixel is formed from image information of the focus detection pixel by the gain correction. If the image signal is formed from the output signal of the focus detection pixel by the gain correction, the image signal which does not relatively depend on the object can be formed. However, if layout density of arrangement of the focus detection pixels is set to be high, a possibility that an image which is obtained in the area where the focus detection pixels are arranged appears to be noiseful is high. In the case of interpolating from the pixels only for image pickup around the focus detection pixel, although the image does not appear to be so noiseful, in order to obtain a good image, it is required to correctly select the interpolation reference pixels.